{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3G1Nx3YLOVaZ"
      },
      "source": [
        "<a target=\"_blank\" href=\"https://colab.research.google.com/github/DeepLabCut/DeepLabCut/blob/main/examples/COLAB/COLAB_DEMO_SuperAnimal.ipynb\">\n",
        "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
        "</a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "23v-XAUNQIPY"
      },
      "source": [
        "# DeepLabCut Model Zoo: SuperAnimal models\n",
        "\n",
        "![alt text](https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/1616492373700-PGOAC72IOB6AUE47VTJX/ke17ZwdGBToddI8pDm48kB8JrdUaZR-OSkKLqWQPp_YUqsxRUqqbr1mOJYKfIPR7LoDQ9mXPOjoJoqy81S2I8N_N4V1vUb5AoIIIbLZhVYwL8IeDg6_3B-BRuF4nNrNcQkVuAT7tdErd0wQFEGFSnBqyW03PFN2MN6T6ry5cmXqqA9xITfsbVGDrg_goIDasRCalqV8R3606BuxERAtDaQ/modelzoo.png?format=1000w)\n",
        "\n",
        "http://modelzoo.deeplabcut.org\n",
        "\n",
        "You can use this notebook to analyze videos with pretrained networks from our model zoo - NO local installation of DeepLabCut is needed!\n",
        "\n",
        "- **What you need:** a video of your favorite dog, cat, human, etc: check the list of currently available models here: http://modelzoo.deeplabcut.org\n",
        "\n",
        "- **What to do:** (1) in the top right corner, click \"CONNECT\". Then, just hit run (play icon) on each cell below and follow the instructions!\n",
        "\n",
        "## **Please consider giving back and labeling a little data to help make each network even better!**\n",
        "\n",
        "We have a WebApp, so no need to install anything, just a few clicks! We'd really appreciate your help! 🙏\n",
        "   \n",
        "https://contrib.deeplabcut.org/\n",
        "\n",
        "\n",
        "- **Note, if you performance is less that you would like:** firstly check the labeled_video parameters (i.e. \"pcutoff\" that will set the video plotting) - see the end of this notebook.\n",
        "- You can also use the model in your own projects locally. Please be sure to cite the papers for the model, i.e., [Ye et al. 2023](https://arxiv.org/abs/2203.07436) 🎉\n",
        "\n",
        "\n",
        "\n",
        "## **Let's get going: install DeepLabCut into COLAB:**\n",
        "\n",
        "*Also, be sure you are connected to a GPU: go to menu, click Runtime > Change Runtime Type > select \"GPU\"*\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "03ylSyQ4O9Ee"
      },
      "outputs": [],
      "source": [
        "!apt update && apt install cuda-11-8\n",
        "!pip install deeplabcut[tf,modelzoo]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TguLMTJpQx1_"
      },
      "source": [
        "## PLEASE, click \"restart runtime\" from the output above before proceeding!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4BejjXKFO2Zg"
      },
      "outputs": [],
      "source": [
        "import deeplabcut\n",
        "import os"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GXf8N4v28Xqo"
      },
      "source": [
        "## Please select a video you want to run SuperAnimal-X on:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xXNMNLe6xEBC"
      },
      "outputs": [],
      "source": [
        "from google.colab import files\n",
        "\n",
        "uploaded = files.upload()\n",
        "for filepath, content in uploaded.items():\n",
        "  print(f'User uploaded file \"{filepath}\" with length {len(content)} bytes')\n",
        "video_path = os.path.abspath(filepath)\n",
        "video_name = os.path.splitext(video_path)[0]\n",
        "\n",
        "# If this cell fails (e.g., when using Safari in place of Google Chrome),\n",
        "# manually upload your video via the Files menu to the left\n",
        "# and define `video_path` yourself with right click > copy path on the video."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A8sDYMa08f62"
      },
      "source": [
        "## Next select the model you want to use, Quadruped or TopViewMouse\n",
        "- See http://modelzoo.deeplabcut.org/ for more details on these models\n",
        "- The pcutoff is for visualization only, namely only keypoints with a value over what you set are shown. 0 is low confidience, 1 is perfect confidience of the model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ge589yC4v9yX"
      },
      "outputs": [],
      "source": [
        "supermodel_name = \"superanimal_quadruped\" #@param [\"superanimal_topviewmouse\", \"superanimal_quadruped\"]\n",
        "pcutoff = 0.3 #@param {type:\"slider\", min:0, max:1, step:0.05}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zsB0pGtj9Luq"
      },
      "source": [
        "## Okay, let's go! 🐭🦓🐻"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "yqcnEVVSQDC0"
      },
      "outputs": [],
      "source": [
        "videotype = os.path.splitext(video_path)[1]\n",
        "scale_list = []\n",
        "\n",
        "deeplabcut.video_inference_superanimal(\n",
        "    [video_path],\n",
        "    supermodel_name,\n",
        "    videotype=videotype,\n",
        "    video_adapt=True,\n",
        "    scale_list=scale_list,\n",
        "    pcutoff=pcutoff,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gPLZSBpD34Mj"
      },
      "source": [
        "## Let's view the video in Colab:\n",
        "- otherwise, you can download and look at the video from the left side of your screen! It will end with _labeled.mp4\n",
        "- If your data doesn't work as well as you'd like, consider fine-tuning our model on your data, changing the pcutoff, changing the scale-range\n",
        "(pick values smaller and larger than your video image input size). See our repo for more details."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "ejFJ1Pbg33i6"
      },
      "outputs": [],
      "source": [
        "from base64 import b64encode\n",
        "from IPython.display import HTML\n",
        "view_video = open(video_name+'DLC_snapshot-1000_labeled.mp4','rb').read()\n",
        "\n",
        "data_url = \"data:video/mp4;base64,\" + b64encode(view_video).decode()\n",
        "HTML(\"\"\"\n",
        "<video width=600 controls>\n",
        "      <source src=\"%s\" type=\"video/mp4\">\n",
        "</video>\n",
        "\"\"\" % data_url)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "gpuClass": "standard",
    "kernelspec": {
      "display_name": "dlc",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.13 | packaged by conda-forge | (main, May 27 2022, 17:01:00) \n[Clang 13.0.1 ]"
    },
    "vscode": {
      "interpreter": {
        "hash": "ef00193d8f29a47f592f520086c931b5dd2a83e8a593fa0efe5afff3c413a788"
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
